Debugging BRX Workflows

Debugging is an essential part of developing with BRX. This guide covers techniques and best practices for identifying and resolving issues in your BRX workflows.

Common Debugging Challenges

When working with BRX, you might encounter several types of issues:

Input/Output Issues: Problems with the data flowing between BRKs
Prompt Engineering Issues: Suboptimal prompts leading to poor LLM responses
Dependency Issues: Problems with the relationships between BRKs
API and Authentication Issues: Problems connecting to the BRX API
Performance Issues: Workflows that are slow or consume too many resources

Enabling Verbose Logging

The first step in debugging BRX workflows is to enable verbose logging:

import BRX from 'brx-node';

// Enable verbose logging
const brx = new BRX('your-api-key', { verbose: true });

With verbose logging enabled, the BRX client will output detailed information about:

WebSocket connections
API requests and responses
BRK execution steps
Error messages and stack traces

Debugging Input/Output Issues

Inspecting Inputs

To debug input issues, inspect the input values before executing a BRK:

// Log all inputs
console.log('BRK Inputs:', JSON.stringify(myBrk.input, null, 2));

// Check specific inputs
if (!myBrk.input['important_field']) {
  console.error('Missing required input: important_field');
}

Validating Input Types

Ensure that inputs have the correct types:

// Validate input types
const validateInputs = (inputs) => {
  const validations = {
    'user_query': (val) => typeof val === 'string' && val.length > 0,
    'max_results': (val) => typeof val === 'number' && val > 0,
    'include_images': (val) => typeof val === 'boolean'
  };
  
  const errors = [];
  
  for (const [key, validator] of Object.entries(validations)) {
    if (inputs[key] !== undefined && !validator(inputs[key])) {
      errors.push(`Invalid value for ${key}: ${inputs[key]}`);
    }
  }
  
  return errors;
};

const validationErrors = validateInputs(myBrk.input);
if (validationErrors.length > 0) {
  console.error('Input validation errors:', validationErrors);
}

Examining Outputs

To debug output issues, examine the results of BRK execution:

const result = await brx.run(myBrk);

// Log the full result
console.log('BRK Result:', JSON.stringify(result, null, 2));

// Check specific output properties
if (!result[0].brxRes.output) {
  console.error('Missing output in result');
}

Debugging Prompt Engineering Issues

Inspecting Prompts

To debug prompt issues, you can inspect the prompts that are sent to the LLM:

// Log the prompt template
console.log('Prompt Template:', myBrk.brxQuery.userSchemaInteract.schemas.get('main_brx_entry_schema').schemaFields.get('prompt').fieldValue);

// Log the rendered prompt (with variables replaced)
// Note: This requires access to the internal BRX engine, which may not be directly accessible

Testing Prompts Independently

You can test prompts independently of BRX to isolate issues:

// Test a prompt with a direct API call to an LLM
const testPrompt = async (prompt) => {
  // Use your preferred LLM API client
  const response = await openai.createCompletion({
    model: 'text-davinci-003',
    prompt,
    max_tokens: 500
  });
  
  return response.choices[0].text;
};

// Test the prompt with sample inputs
const samplePrompt = `Generate a product description for the following:
Product Name: Ultra Comfort Ergonomic Chair
Product Category: Office Furniture
Key Features: Adjustable height, lumbar support, breathable mesh, 360-degree swivel
Target Audience: Office workers, remote professionals`;

const result = await testPrompt(samplePrompt);
console.log('Test Result:', result);

Debugging prompts often involves iterative refinement:

Start with a simple prompt
Test it with various inputs
Analyze the results
Refine the prompt
Repeat until the results meet your expectations

Debugging Dependency Issues

Visualizing Dependencies

To debug dependency issues, visualize the dependency graph:

// Simple function to log the dependency graph
const logDependencyGraph = (brk) => {
  console.log(`BRK: ${brk.brxQuery.userSchemaInteract.mainBrxId}`);
  
  const schemas = brk.brxQuery.userSchemaInteract.schemas;
  schemas.forEach((schema, key) => {
    console.log(`  Schema: ${key} (${schema.brxId})`);
    
    // Log dependencies if available
    if (brk.brxQuery.dependantBrxIds) {
      brk.brxQuery.dependantBrxIds.forEach((depId, depKey) => {
        console.log(`    Dependency: ${depKey} -> ${depId}`);
      });
    }
  });
};

logDependencyGraph(myBrk);

Testing Dependencies Individually

Test each BRK in the dependency chain individually:

// Test each BRK in isolation
const testDependencyChain = async () => {
  // Test the first BRK
  const extractBrkSchema = await brx.get('extract-brk-id');
  const extractBrk = new BRK(extractBrkSchema);
  extractBrk.input['text'] = 'Sample text to extract data from';
  const extractResult = await brx.run(extractBrk);
  console.log('Extract Result:', extractResult[0].brxRes.output);
  
  // Test the second BRK with the output of the first
  const analyzeBrkSchema = await brx.get('analyze-brk-id');
  const analyzeBrk = new BRK(analyzeBrkSchema);
  analyzeBrk.input['data'] = extractResult[0].brxRes.output;
  const analyzeResult = await brx.run(analyzeBrk);
  console.log('Analyze Result:', analyzeResult[0].brxRes.output);
  
  // Continue for each BRK in the chain
};

testDependencyChain();

Debugging API and Authentication Issues

Testing API Connectivity

To debug API connectivity issues:

// Test API connectivity
const testApiConnectivity = async () => {
  try {
    // Try a simple API call
    const response = await fetch('https://api.brx.ai', {
      method: 'GET',
      headers: {
        'Content-Type': 'application/json'
      }
    });
    
    console.log('API Status:', response.status);
    console.log('API Response:', await response.text());
    
    return response.ok;
  } catch (error) {
    console.error('API Connectivity Error:', error);
    return false;
  }
};

testApiConnectivity();

Verifying Authentication

To debug authentication issues:

// Test authentication
const testAuthentication = async (apiKey) => {
  try {
    // Try an authenticated API call
    const response = await fetch('https://api.brx.ai/list_brx', {
      method: 'POST',
      headers: {
        'Content-Type': 'application/json',
        'key': apiKey
      },
      body: JSON.stringify({})
    });
    
    const data = await response.json();
    
    if (data.httpResponse && data.httpResponse.isError) {
      console.error('Authentication Error:', data.httpResponse.statusMsg);
      return false;
    }
    
    console.log('Authentication Successful');
    return true;
  } catch (error) {
    console.error('Authentication Error:', error);
    return false;
  }
};

testAuthentication('your-api-key');

Debugging Performance Issues

Measuring Execution Time

To debug performance issues, measure execution time:

// Measure BRK execution time
const measureExecutionTime = async (brk) => {
  const startTime = Date.now();
  
  const result = await brx.run(brk);
  
  const endTime = Date.now();
  const executionTime = endTime - startTime;
  
  console.log(`BRK Execution Time: ${executionTime}ms`);
  
  return { result, executionTime };
};

const { result, executionTime } = await measureExecutionTime(myBrk);

Profiling BRK Execution

For more detailed performance analysis, profile the execution of each BRK in a workflow:

// Profile a workflow with multiple BRKs
const profileWorkflow = async () => {
  const timings = {};
  
  // Execute first BRK
  const startTime1 = Date.now();
  const result1 = await brx.run(brk1);
  timings.brk1 = Date.now() - startTime1;
  
  // Execute second BRK
  const startTime2 = Date.now();
  const result2 = await brx.run(brk2);
  timings.brk2 = Date.now() - startTime2;
  
  // Execute third BRK
  const startTime3 = Date.now();
  const result3 = await brx.run(brk3);
  timings.brk3 = Date.now() - startTime3;
  
  console.log('Workflow Timings:', timings);
  console.log('Total Execution Time:', Object.values(timings).reduce((a, b) => a + b, 0));
  
  return { result1, result2, result3, timings };
};

profileWorkflow();

Advanced Debugging Techniques

Using Callbacks for Real-Time Debugging

BRX supports callbacks that can be used for real-time debugging:

// Use callbacks for real-time debugging
const results = [];

await brx.run(myBrk, (result) => {
  console.log(`Received result from ${result.brxName}`);
  console.log('Result:', JSON.stringify(result.brxRes, null, 2));
  
  results.push(result);
});

console.log('All Results:', results);

Creating Debug BRKs

You can create special BRKs specifically for debugging:

// Create a debug BRK
const createDebugBrk = async () => {
  const debugBrkRequest = {
    modifyBrxMode: 'CREATE',
    brx: {
      brxId: 'debug-brk',
      brxName: 'Debug BRK',
      description: 'A BRK for debugging purposes',
      prompt: {
        prompt: new Map([
          ['main', `
            Debug Information:
            
            Input: {{input_json}}
            
            Please analyze this input and provide the following:
            1. A summary of the input data
            2. Any potential issues or inconsistencies
            3. Suggestions for improvement
          `]
        ])
      },
      processParams: {
        processType: 0
      },
      dependantBrxIds: new Map([
        ['main_brx_entry_schema', 'debug-brk']
      ])
    },
    schema: {
      schemaFields: new Map([
        ['input_json', {
          fieldValueDataType: 'string',
          fieldValue: ''
        }]
      ]),
      brxName: 'Debug BRK',
      brxId: 'debug-brk'
    }
  };
  
  const result = await brx.create(debugBrkRequest);
  console.log('Debug BRK created:', result);
  
  return result;
};

// Use the debug BRK
const debugInput = async (input) => {
  const debugBrkSchema = await brx.get('debug-brk');
  const debugBrk = new BRK(debugBrkSchema);
  
  debugBrk.input['input_json'] = JSON.stringify(input, null, 2);
  
  const result = await brx.run(debugBrk);
  console.log('Debug Result:', result[0].brxRes.output);
  
  return result;
};

// Create the debug BRK (only need to do this once)
await createDebugBrk();

// Debug an input
await debugInput(myBrk.input);

Mocking Dependencies

For complex workflows, you can mock dependencies to isolate issues:

// Mock a dependency BRK
const mockDependency = async (dependencyBrkId, mockOutput) => {
  // Create a mock BRK with the same ID but simplified behavior
  const mockBrkRequest = {
    modifyBrxMode: 'CREATE',
    brx: {
      brxId: `mock-${dependencyBrkId}`,
      brxName: `Mock ${dependencyBrkId}`,
      description: 'A mock BRK for testing',
      prompt: {
        prompt: new Map([
          ['main', `Return the following mock output: ${mockOutput}`]
        ])
      },
      processParams: {
        processType: 0
      },
      dependantBrxIds: new Map([
        ['main_brx_entry_schema', `mock-${dependencyBrkId}`]
      ])
    },
    schema: {
      schemaFields: new Map([]),
      brxName: `Mock ${dependencyBrkId}`,
      brxId: `mock-${dependencyBrkId}`
    }
  };
  
  const result = await brx.create(mockBrkRequest);
  console.log(`Mock BRK created for ${dependencyBrkId}:`, result);
  
  return result;
};

// Create a mock dependency
await mockDependency('data-extraction-brk', '{"extracted_data": "This is mock extracted data"}');

// Update the main BRK to use the mock dependency
// This would require modifying the dependantBrxIds map

Debugging Tools and Resources

BRX Dashboard

The BRX Dashboard provides tools for monitoring and debugging your BRKs:

View BRK execution history
Inspect BRK inputs and outputs
Monitor API usage and performance
Manage API keys and permissions

Logging Services

Consider using a logging service for more advanced debugging:

// Example using Winston for logging
const winston = require('winston');

const logger = winston.createLogger({
  level: 'debug',
  format: winston.format.json(),
  defaultMeta: { service: 'brx-app' },
  transports: [
    new winston.transports.File({ filename: 'error.log', level: 'error' }),
    new winston.transports.File({ filename: 'combined.log' })
  ]
});

// Log BRK execution
const logBrkExecution = async (brk) => {
  logger.info('Executing BRK', {
    brkId: brk.brxQuery.userSchemaInteract.mainBrxId,
    inputs: brk.input
  });
  
  try {
    const result = await brx.run(brk);
    
    logger.info('BRK execution successful', {
      brkId: brk.brxQuery.userSchemaInteract.mainBrxId,
      resultCount: result.length
    });
    
    return result;
  } catch (error) {
    logger.error('BRK execution failed', {
      brkId: brk.brxQuery.userSchemaInteract.mainBrxId,
      error: error.message,
      stack: error.stack
    });
    
    throw error;
  }
};

const result = await logBrkExecution(myBrk);

Error Tracking Services

For production applications, consider using an error tracking service:

// Example using Sentry for error tracking
const Sentry = require('@sentry/node');

Sentry.init({
  dsn: 'your-sentry-dsn',
  environment: process.env.NODE_ENV
});

// Wrap BRK execution with error tracking
const trackBrkExecution = async (brk) => {
  try {
    const result = await brx.run(brk);
    return result;
  } catch (error) {
    Sentry.captureException(error, {
      tags: {
        brkId: brk.brxQuery.userSchemaInteract.mainBrxId
      },
      extra: {
        inputs: brk.input
      }
    });
    
    throw error;
  }
};

const result = await trackBrkExecution(myBrk);

Best Practices for Debugging

Incremental Development

Build and test your BRX workflows incrementally:

Start with a simple BRK and verify it works
Add dependencies one at a time
Test each addition thoroughly
Refine and optimize as needed

Comprehensive Testing

Implement comprehensive testing for your BRX workflows:

Unit tests for individual BRKs
Integration tests for BRK dependencies
End-to-end tests for complete workflows
Performance tests for critical paths

Documentation

Document your debugging process:

Record issues and their solutions
Document common patterns and anti-patterns
Create debugging guides for your team
Share lessons learned and best practices

Monitoring

Implement monitoring for production BRX applications:

Monitor API usage and rate limits
Track execution times and performance
Set up alerts for errors and failures
Analyze usage patterns to identify optimization opportunities

By following these debugging techniques and best practices, you can effectively identify and resolve issues in your BRX workflows, leading to more reliable and efficient applications.

Get Started

Core Concepts

SDK Reference

Debugging

Debugging BRX Workflows

Common Debugging Challenges

Enabling Verbose Logging

Debugging Input/Output Issues

Inspecting Inputs

Validating Input Types

Examining Outputs

Debugging Prompt Engineering Issues

Inspecting Prompts

Testing Prompts Independently

Iterative Prompt Refinement

Debugging Dependency Issues

Visualizing Dependencies

Testing Dependencies Individually

Debugging API and Authentication Issues

Testing API Connectivity

Verifying Authentication

Debugging Performance Issues

Measuring Execution Time

Profiling BRK Execution

Advanced Debugging Techniques

Using Callbacks for Real-Time Debugging

Creating Debug BRKs

Mocking Dependencies

Debugging Tools and Resources

BRX Dashboard

Logging Services

Error Tracking Services

Best Practices for Debugging

Incremental Development

Comprehensive Testing

Documentation

Monitoring

Get Started

Core Concepts

SDK Reference

​Debugging BRX Workflows

​Common Debugging Challenges

​Enabling Verbose Logging

​Debugging Input/Output Issues

​Inspecting Inputs

​Validating Input Types

​Examining Outputs

​Debugging Prompt Engineering Issues

​Inspecting Prompts

​Testing Prompts Independently

​Iterative Prompt Refinement

​Debugging Dependency Issues

​Visualizing Dependencies

​Testing Dependencies Individually

​Debugging API and Authentication Issues

​Testing API Connectivity

​Verifying Authentication

​Debugging Performance Issues

​Measuring Execution Time

​Profiling BRK Execution

​Advanced Debugging Techniques

​Using Callbacks for Real-Time Debugging

​Creating Debug BRKs

​Mocking Dependencies

​Debugging Tools and Resources

​BRX Dashboard

​Logging Services

​Error Tracking Services

​Best Practices for Debugging

​Incremental Development

​Comprehensive Testing

​Documentation

​Monitoring

Debugging BRX Workflows

Common Debugging Challenges

Enabling Verbose Logging

Debugging Input/Output Issues

Inspecting Inputs

Validating Input Types

Examining Outputs

Debugging Prompt Engineering Issues

Inspecting Prompts

Testing Prompts Independently

Iterative Prompt Refinement

Debugging Dependency Issues

Visualizing Dependencies

Testing Dependencies Individually

Debugging API and Authentication Issues

Testing API Connectivity

Verifying Authentication

Debugging Performance Issues

Measuring Execution Time

Profiling BRK Execution

Advanced Debugging Techniques

Using Callbacks for Real-Time Debugging

Creating Debug BRKs

Mocking Dependencies

Debugging Tools and Resources

BRX Dashboard

Logging Services

Error Tracking Services

Best Practices for Debugging

Incremental Development

Comprehensive Testing

Documentation

Monitoring